The present disclosure generally relates to a computerized device including a touchpad installed on the back panel or other portion of the body other than the display screen. More particularly, the present disclosure relates to a method and graphical user interface that enables the user to see the user's finger position and motion from the back or other portion of the device, superimposed on a keyboard layout on the display screen. This makes it easier for a user to input keystrokes and mouse actions from a touchpad that is installed on the back panel or other portion of a handheld device. In an embodiment, the user can also control and manipulate a virtual keyboard shown in the display screen. Although embodiments of the invention are described with reference to a handheld computerized device by way of an example, it is understood that the invention is not limited by the type of computerized device or system.
Handheld computerized devices, i.e., devices including microprocessors and sophisticated displays) such as cell phones, personal digital assistants (PDA), game devices, tabletPCs, such as iPad, wearable computerized devices, and the like, are playing a more and more important role in everyday life, and are becoming more and more indispensable. With the advance of technology, and improvements in the handheld computerized devices' processing power, both function, and memory space is increasing at an amazing pace. Meanwhile the size of the handheld computerized devices continues to get smaller and smaller making the touchpad and display on the device smaller and more challenging to use.
To meet the challenge of a smaller device display and touchpad, the designers of handheld computerized devices typically use two approaches. One approach is to make the keyboard keys smaller and smaller, miniaturizing the keys. Additionally the keyboard keys may be given multiple functions—i.e. overloaded, and more complex function keyboard keys may be introduced as well.
The other approach is to use touch screen keyboards, or so called “soft keys”, on the front panel. Here a user may use a stylus pen or finger to select the soft keys through a graphical user interface. However due to the optical illusions introduced by the display screen, and the fact that the user's fingers often are on top of the various display screen soft keys, hence blocking the keys from direct viewing, the soft keys should not be too small. Another problem is that when the soft keys are too small, often a single finger press will activate multiple keys. As a result, the designer may have to divide the keys into different groups and hierarchies, and just display a small number of keys at a time on the screen.
Both current approaches have some drawbacks: the user input area may occupy a significant portion of the front panel, and the user input process, although requiring a large amount of user attention to operate, still is very error prone.
Often a user may use one hand to hold the handheld computerized device, and use the other hand to input data, thus, occupying both hands. A user will often have to go through a long sequence of key strokes, and switch back and forth among different user interface screens, in order to complete a fairly simple input. As a result, there is a significant learning curve for a user to learn the overloaded keys, function keys, key grouping, and key hierarchies in order to operate the handheld computerized devices efficiently.
Previous designs including sensors on the back of the device and representations of the user's fingers on the front of the device, however, this work failed to adequately describe a procedure by which the indicia of the user's fingers or hands are displayed on the display panel.
Systems have been described in which image sensors would obtain an image of the user's fingers while operating the device, and use this image data to better determine which real or virtual keys the user's fingers were striking Such methods rely, however, on image sensors that are positioned in such a way as to be capable of viewing the tips of the user's fingers. This type of image sensor placement is often difficult to implement on many types of handheld user computerized devices. Another drawback of the previous image sensor approach is that it is difficult to implement in low light situations. This approach may also be difficult to implement in situations where there is limited smooth and flat desk or table space.